fix: disable Anthropic thinking to fix tool chain crash#415
fix: disable Anthropic thinking to fix tool chain crash#415sweetmantech merged 6 commits intotestfrom
Conversation
Anthropic's API rejects requests that combine extended thinking with forced tool_choice. The ToolLoopAgent enables thinking globally, and prepareStep forces toolChoice for every tool chain step (create_new_artist, create_release_report). This causes "Thinking may not be enabled when tool_choice forces tool use" on any Anthropic model. The Vercel AI SDK does not support overriding providerOptions per-step (open issue vercel/ai#11761), so the fix must be at the constructor level. Disables Anthropic thinking until the SDK adds per-step providerOptions support or an alternative approach is implemented. Made-with: Cursor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughExpanded Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
🚥 Pre-merge checks | ❌ 1❌ Failed checks (1 warning)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
lib/agents/generalAgent/getGeneralAgent.ts (1)
24-90: Pre-existing: Function exceeds length guidelines (~66 lines).This isn't introduced by your PR, but the
getGeneralAgentfunction violates the 50-line guideline forlib/**/*.ts. When time permits, consider extracting logical segments into focused helper functions:
fetchArtistContext(artistId)— lines 30-37buildInstructionsWithImages(body, ...)— lines 40-51buildAgentProviderOptions()— lines 67-81This would improve testability and adhere to SRP. Not blocking for this fix.
As per coding guidelines: "Keep functions under 50 lines" and "Flag functions longer than 20 lines."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@lib/agents/generalAgent/getGeneralAgent.ts` around lines 24 - 90, The getGeneralAgent function is over the 50-line guideline; extract the logical blocks into small helpers: implement fetchArtistContext(artistId) to encapsulate selectAccountInfo and getKnowledgeBaseText and return { artistInstruction, knowledgeBaseText }, implement buildInstructionsWithImages(body, baseSystemPrompt) to create baseSystemPrompt via getSystemPrompt, call extractImageUrlsFromMessages and buildSystemPromptWithImages and return instructions, and implement buildAgentProviderOptions() to return the providerOptions object used in new ToolLoopAgent; then update getGeneralAgent to call these helpers (keep ToolLoopAgent construction but use buildAgentProviderOptions(), fetchArtistContext(artistId), and buildInstructionsWithImages(body, baseSystemPrompt)) so the top-level function falls under the line limit while preserving behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@lib/agents/generalAgent/getGeneralAgent.ts`:
- Around line 24-90: The getGeneralAgent function is over the 50-line guideline;
extract the logical blocks into small helpers: implement
fetchArtistContext(artistId) to encapsulate selectAccountInfo and
getKnowledgeBaseText and return { artistInstruction, knowledgeBaseText },
implement buildInstructionsWithImages(body, baseSystemPrompt) to create
baseSystemPrompt via getSystemPrompt, call extractImageUrlsFromMessages and
buildSystemPromptWithImages and return instructions, and implement
buildAgentProviderOptions() to return the providerOptions object used in new
ToolLoopAgent; then update getGeneralAgent to call these helpers (keep
ToolLoopAgent construction but use buildAgentProviderOptions(),
fetchArtistContext(artistId), and buildInstructionsWithImages(body,
baseSystemPrompt)) so the top-level function falls under the line limit while
preserving behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 57c94250-3bdd-413f-9504-493b2b9bd160
⛔ Files ignored due to path filters (1)
lib/agents/generalAgent/__tests__/getGeneralAgent.test.tsis excluded by!**/*.test.*,!**/__tests__/**and included bylib/**
📒 Files selected for processing (1)
lib/agents/generalAgent/getGeneralAgent.ts
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
No issues found across 2 files
Confidence score: 5/5
- Automated review surfaced no issues in the provided summaries.
- No files require special attention.
Requires human review: This is a global configuration change to the AI agent's behavior that disables a feature (Anthropic thinking) to resolve a crash, requiring a human review of the trade-off.
Architecture diagram
sequenceDiagram
participant Client
participant Agent as General Agent (getGeneralAgent)
participant SDK as Vercel AI SDK / ToolLoop
participant Anthropic as Anthropic API
Note over Client,Anthropic: Request Flow with Anthropic Model (e.g., Claude 3.7)
Client->>Agent: POST /api/chat (Request Body)
Agent->>Agent: CHANGED: Initialize providerOptions<br/>Set anthropic.thinking = "disabled"
Note right of Agent: Prevents crash when specific tools are forced
Agent->>SDK: Start Tool Loop
SDK->>SDK: getPrepareStepResult()
Note right of SDK: Logic returns toolChoice: { type: 'tool' }
SDK->>Anthropic: POST /v1/messages
Note right of Anthropic: Request contains:<br/>1. tool_choice: { type: 'tool', ... }<br/>2. thinking: { type: 'disabled' }
alt Success Path
Anthropic-->>SDK: 200 OK (Tool Call/Response)
SDK-->>Client: Streamed Response
else Previous Crash Path (Pre-fix)
Note over SDK,Anthropic: If thinking was 'enabled' and tool_choice was forced
Anthropic-->>SDK: 400 Bad Request (Invalid combination)
SDK-->>Client: 500 Error Crash
end
Note over Agent,SDK: Note: OpenAI reasoningEffort and Google thinkingConfig<br/>remain unchanged and active.
…ing conflict Instead of disabling Anthropic thinking globally, tool chain steps now switch to openai/gpt-5-mini (via TOOL_CHAIN_FALLBACK_MODEL) when no specific model is mapped in TOOL_MODEL_MAP. This preserves extended thinking for regular Anthropic conversations while ensuring forced toolChoice works during tool chain execution. Made-with: Cursor
There was a problem hiding this comment.
1 issue found across 5 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="lib/agents/generalAgent/getGeneralAgent.ts">
<violation number="1">
P0: Custom agent: **Flag AI Slop and Fabricated Changes**
This change **enables** Anthropic thinking (`type: "enabled", budgetTokens: 12000`) but the PR title, description, and summary all claim it **disables** thinking. The code does the exact opposite of its stated intent.
Per the PR's own problem statement, enabling thinking is what *causes* the crash (`"Thinking may not be enabled when tool_choice forces tool use"`). The previous code already had thinking disabled — this change reintroduces the crash rather than fixing it.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.
Made-with: Cursor
There was a problem hiding this comment.
0 issues found across 2 files (changes from recent commits).
Requires human review: The model identifier 'openai/gpt-5.4-mini' appears to be a typo or invalid, which would cause tool chains to crash in production. There is also a mismatch between the PR description and diff.
There was a problem hiding this comment.
🧹 Nitpick comments (1)
lib/chat/toolChains/getPrepareStepResult.ts (1)
60-60: Align thePrepareStepResulttype with Line 60 behavior.Line 60 always sets
result.model, but the shared type still marksmodelas optional. Tightening that contract will improve downstream type safety and intent clarity.As per coding guidelines `lib/**/*.ts`: Use TypeScript for type safety.♻️ Proposed contract alignment
--- a/lib/chat/toolChains/toolChains.ts +++ b/lib/chat/toolChains/toolChains.ts @@ export type PrepareStepResult = { toolChoice?: { type: "tool"; toolName: string }; - model?: LanguageModel; + model: LanguageModel; system?: string; messages?: ModelMessage[]; };🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@lib/chat/toolChains/getPrepareStepResult.ts` at line 60, The PrepareStepResult type currently marks "model" as optional but getPrepareStepResult.ts (in the getPrepareStepResult function) always assigns result.model (line with TOOL_MODEL_MAP[nextToolItem.toolName] || TOOL_CHAIN_FALLBACK_MODEL), so update the shared PrepareStepResult type to make "model" required (non-optional), then fix any callsites/types that assumed it could be undefined (e.g., places referencing PrepareStepResult, the getPrepareStepResult return type, and any tests) to reflect the tightened contract; ensure exports/imports remain consistent and run type checks to catch and correct any mismatches.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@lib/chat/toolChains/getPrepareStepResult.ts`:
- Line 60: The PrepareStepResult type currently marks "model" as optional but
getPrepareStepResult.ts (in the getPrepareStepResult function) always assigns
result.model (line with TOOL_MODEL_MAP[nextToolItem.toolName] ||
TOOL_CHAIN_FALLBACK_MODEL), so update the shared PrepareStepResult type to make
"model" required (non-optional), then fix any callsites/types that assumed it
could be undefined (e.g., places referencing PrepareStepResult, the
getPrepareStepResult return type, and any tests) to reflect the tightened
contract; ensure exports/imports remain consistent and run type checks to catch
and correct any mismatches.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 0b5bdb6e-f0c7-45e5-8245-4fd662a4c800
⛔ Files ignored due to path filters (1)
lib/chat/toolChains/__tests__/getPrepareStepResult.test.tsis excluded by!**/*.test.*,!**/__tests__/**and included bylib/**
📒 Files selected for processing (2)
lib/chat/toolChains/getPrepareStepResult.tslib/chat/toolChains/toolChains.ts
Testing InstructionsThe preview deployment is live at: How to test
What to verify
How the fix worksTool chain steps now switch to Reset after testingsessionStorage.removeItem('recoup_api_override'); |
Test Results ✅Tested against the preview deployment via curl with What happened
Before the fixThis would have crashed with:
After the fix
Test command usedcurl -s -N -X POST \
-H "Content-Type: application/json" \
-H "x-api-key: $RECOUP_TEST_API_KEY" \
"https://recoup-api-git-fix-disable-anthropic-cb22a1-recoupable-ad724970.vercel.app/api/chat" \
-d '{"prompt":"Create a new artist named Xavier Pages","model":"anthropic/claude-sonnet-4.5"}' |
Made-with: Cursor
| if (model) { | ||
| result.model = model; | ||
| } | ||
| result.model = TOOL_MODEL_MAP[nextToolItem.toolName] || TOOL_CHAIN_FALLBACK_MODEL; |
There was a problem hiding this comment.
KISS - Why do we need a fallback?
- Why not use the existing
toolNamelogic?
lib/chat/toolChains/toolChains.ts
Outdated
|
|
||
| // Forced toolChoice is incompatible with Anthropic extended thinking. | ||
| // Tool chain steps use this model unless overridden by TOOL_MODEL_MAP. | ||
| export const TOOL_CHAIN_FALLBACK_MODEL: LanguageModel = "openai/gpt-5.4-mini"; |
There was a problem hiding this comment.
KISS - remove the fallback and use the existing TOOL_MODEL_MAP.
…l chain tools Addresses code review feedback — use existing TOOL_MODEL_MAP pattern instead of a separate fallback constant. All tool chain tools now have explicit model entries in TOOL_MODEL_MAP. Made-with: Cursor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
lib/chat/toolChains/toolChains.ts (1)
18-37: Consider extracting the repeated model string to a constant for DRY compliance.The string
"openai/gpt-5.4-mini"is repeated 15 times across the mapping. Extracting it to a constant would reduce duplication and make future model changes a single-line edit.That said, I recognize explicit per-tool mappings provide clarity and allow granular overrides—so this is a recommended improvement rather than a blocker.
♻️ Proposed refactor to extract constant
+const DEFAULT_TOOL_CHAIN_MODEL = "openai/gpt-5.4-mini" as LanguageModel; + // Forced toolChoice is incompatible with Anthropic extended thinking. // Every tool used in a chain must have a model here to avoid the conflict. export const TOOL_MODEL_MAP: Record<string, LanguageModel> = { update_account_info: "gemini-2.5-pro", - get_spotify_search: "openai/gpt-5.4-mini", - update_artist_socials: "openai/gpt-5.4-mini", - artist_deep_research: "openai/gpt-5.4-mini", - spotify_deep_research: "openai/gpt-5.4-mini", - get_artist_socials: "openai/gpt-5.4-mini", - get_spotify_artist_top_tracks: "openai/gpt-5.4-mini", - get_spotify_artist_albums: "openai/gpt-5.4-mini", - get_spotify_album: "openai/gpt-5.4-mini", - search_web: "openai/gpt-5.4-mini", - generate_txt_file: "openai/gpt-5.4-mini", - create_segments: "openai/gpt-5.4-mini", - youtube_login: "openai/gpt-5.4-mini", - web_deep_research: "openai/gpt-5.4-mini", - create_knowledge_base: "openai/gpt-5.4-mini", - send_email: "openai/gpt-5.4-mini", + get_spotify_search: DEFAULT_TOOL_CHAIN_MODEL, + update_artist_socials: DEFAULT_TOOL_CHAIN_MODEL, + artist_deep_research: DEFAULT_TOOL_CHAIN_MODEL, + spotify_deep_research: DEFAULT_TOOL_CHAIN_MODEL, + get_artist_socials: DEFAULT_TOOL_CHAIN_MODEL, + get_spotify_artist_top_tracks: DEFAULT_TOOL_CHAIN_MODEL, + get_spotify_artist_albums: DEFAULT_TOOL_CHAIN_MODEL, + get_spotify_album: DEFAULT_TOOL_CHAIN_MODEL, + search_web: DEFAULT_TOOL_CHAIN_MODEL, + generate_txt_file: DEFAULT_TOOL_CHAIN_MODEL, + create_segments: DEFAULT_TOOL_CHAIN_MODEL, + youtube_login: DEFAULT_TOOL_CHAIN_MODEL, + web_deep_research: DEFAULT_TOOL_CHAIN_MODEL, + create_knowledge_base: DEFAULT_TOOL_CHAIN_MODEL, + send_email: DEFAULT_TOOL_CHAIN_MODEL, };🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@lib/chat/toolChains/toolChains.ts` around lines 18 - 37, TOOL_MODEL_MAP repeats the literal "openai/gpt-5.4-mini" many times; define a single constant (e.g., DEFAULT_TOOL_MODEL = "openai/gpt-5.4-mini") and replace the repeated string values in TOOL_MODEL_MAP with that constant while leaving unique entries like update_account_info: "gemini-2.5-pro" untouched so future model changes only require editing the constant.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@lib/chat/toolChains/toolChains.ts`:
- Around line 18-37: TOOL_MODEL_MAP repeats the literal "openai/gpt-5.4-mini"
many times; define a single constant (e.g., DEFAULT_TOOL_MODEL =
"openai/gpt-5.4-mini") and replace the repeated string values in TOOL_MODEL_MAP
with that constant while leaving unique entries like update_account_info:
"gemini-2.5-pro" untouched so future model changes only require editing the
constant.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: bdea683b-d278-46f1-8be3-41aaf1bf0479
📒 Files selected for processing (2)
lib/chat/toolChains/getPrepareStepResult.tslib/chat/toolChains/toolChains.ts
💤 Files with no reviewable changes (1)
- lib/chat/toolChains/getPrepareStepResult.ts
There was a problem hiding this comment.
1 issue found across 3 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="lib/chat/toolChains/toolChains.ts">
<violation number="1" location="lib/chat/toolChains/toolChains.ts:22">
P1: Custom agent: **Flag AI Slop and Fabricated Changes**
15 map entries hardcoded to the same `"openai/gpt-5.4-mini"` value replace the removed `TOOL_CHAIN_FALLBACK_MODEL` default. This is duplicated configuration that's fragile: any new tool added to a chain without a map entry here silently regresses to the original crash (the consumer in `getPrepareStepResult.ts` only sets the model `if (model)` — there is no fallback).
Restore a default/fallback constant and keep `TOOL_MODEL_MAP` only for overrides (like `update_account_info → gemini-2.5-pro`).</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.
| update_account_info: "gemini-2.5-pro", | ||
| // Add other tools that need specific models here | ||
| // e.g., create_segments: "gpt-4-turbo", | ||
| get_spotify_search: "openai/gpt-5.4-mini", |
There was a problem hiding this comment.
P1: Custom agent: Flag AI Slop and Fabricated Changes
15 map entries hardcoded to the same "openai/gpt-5.4-mini" value replace the removed TOOL_CHAIN_FALLBACK_MODEL default. This is duplicated configuration that's fragile: any new tool added to a chain without a map entry here silently regresses to the original crash (the consumer in getPrepareStepResult.ts only sets the model if (model) — there is no fallback).
Restore a default/fallback constant and keep TOOL_MODEL_MAP only for overrides (like update_account_info → gemini-2.5-pro).
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At lib/chat/toolChains/toolChains.ts, line 22:
<comment>15 map entries hardcoded to the same `"openai/gpt-5.4-mini"` value replace the removed `TOOL_CHAIN_FALLBACK_MODEL` default. This is duplicated configuration that's fragile: any new tool added to a chain without a map entry here silently regresses to the original crash (the consumer in `getPrepareStepResult.ts` only sets the model `if (model)` — there is no fallback).
Restore a default/fallback constant and keep `TOOL_MODEL_MAP` only for overrides (like `update_account_info → gemini-2.5-pro`).</comment>
<file context>
@@ -16,12 +16,24 @@ export type PrepareStepResult = {
+// Every tool used in a chain must have a model here to avoid the conflict.
export const TOOL_MODEL_MAP: Record<string, LanguageModel> = {
update_account_info: "gemini-2.5-pro",
+ get_spotify_search: "openai/gpt-5.4-mini",
+ update_artist_socials: "openai/gpt-5.4-mini",
+ artist_deep_research: "openai/gpt-5.4-mini",
</file context>
Test Results — After Code Review Refactor ✅Tested latest commit ( Tool chain execution trace
All 10 steps executed without error. Anthropic thinking works on step 1, then |
Verification Test Results ✅Tested both prod (current) and preview (this PR) with the same request: curl -s -N -X POST \
-H "Content-Type: application/json" \
-H "x-api-key: $API_KEY" \
"$ENDPOINT/api/chat" \
-d '{"prompt":"Create a new artist named Test Artist","model":"anthropic/claude-sonnet-4.5"}'Prod (before fix) ❌
Preview (with fix) ✅
Bug confirmed on prod, fix confirmed on preview. 🤖 Generated with Claude Code |
3 participants
Summary
thinking: { type: "disabled" }) in theToolLoopAgentconstructor to prevent crash when tool chains forcetoolChoiceProblem
When a user selects an Anthropic model (Claude Opus 4.5, Claude Sonnet 4.5) and triggers a tool chain (
create_new_artist,create_release_report), the API crashes with:This happens because:
getGeneralAgent.tsenablesthinking: { type: "enabled", budgetTokens: 12000 }globallygetPrepareStepResult.tsreturnstoolChoice: { type: "tool", toolName: "..." }for each chain steptoolChoice: "auto"or"none"are allowed with thinkingWhy this approach
The Vercel AI SDK's
PrepareStepResulttype does not includeproviderOptions, so thinking cannot be disabled per-step. This is a known SDK limitation (open feature request with the exact same use case). The constructor-level disable is the only guaranteed fix.Trade-off
Anthropic model users lose extended thinking for all conversations (not just tool chain steps). This is acceptable because:
openai/gpt-5-mini— majority of users are unaffectedreasoningEffortand Google'sthinkingConfigare unaffectedTest plan
getGeneralAgent.test.ts— 25 tests passMade with Cursor
Summary by cubic
Fixes tool chain crashes caused by Anthropic extended thinking with forced
toolChoiceby assigning non-Anthropic models to every chain step viaTOOL_MODEL_MAP. Tool chains now work with Claude 4.5, while extended thinking remains enabled for normal chats.TOOL_MODEL_MAPfor all chain tools (mostlyopenai/gpt-5.4-mini;update_account_infousesgemini-2.5-pro).getPrepareStepResultsets the mappedmodelfor each step, avoiding “Thinking may not be enabled when tool_choice forces tool use”.Written for commit 40a1216. Summary will update on new commits.
Summary by CodeRabbit